Practice Feedback Interventions: 15 Suggestions for Optimizing Effectiveness

Electronic practice data are increasingly being used to provide feedback to encourage practice improvement. However, evidence suggests that despite decades of experience, the effects of such interventions vary greatly and are not improving over time. Guidance on providing more effective feedback does exist, but it is distributed across a wide range of disciplines and theoretical perspectives. Through expert interviews; systematic reviews; and experience with providing, evaluating, and receiving practice feedback, 15 suggestions that are believed to be associated with effective feedback interventions have been identified. These suggestions are intended to provide practical guidance to quality improvement professionals, information technology developers, educators, administrators, and practitioners who receive such interventions. Designing interventions with these suggestions in mind should improve their effect, and studying the mechanisms underlying these suggestions will advance a stagnant literature

Exploring the divergence between self-assessment and self-monitoring

Many models of professional self-regulation call upon individual practitioners to take responsibility both for identifying the limits of their own skills and for redressing their identified limits through continuing professional development activities. Despite these expectations, a considerable literature in the domain of self-assessment has questioned the ability of the self-regulating professional to enact this process effectively. In response, authors have recently suggested that the construction of self-assessment as represented in the self-regulation literature is, itself, problematic. In this paper we report a pair of studies that examine the relationship between self-assessment (a global judgment of one’s ability in a particular domain) and self-monitoring (a moment-by-moment awareness of the likelihood that one maintains the skill/knowledge to act in a particular situation). These studies reveal that, despite poor correlations between performance and self-assessments (consistent with what is typically seen in the self-assessment literature), participant performance was strongly related to several measures of self-monitoring including: the decision to answer or defer responding to a question, the amount of time required to make that decision to answer or defer, and the confidence expressed in an answer when provided. This apparent divergence between poor overall self-assessment and effective self-monitoring is considered in terms of how the findings might inform our understanding of the cognitive mechanisms yielding both self-monitoring judgments and self-assessments and how that understanding might be used to better direct education and learning efforts

A generalizability study of the medical judgment vignettes interview to assess students' noncognitive attributes for medical school

<p>Abstract</p> <p>Background</p> <p>Although the reliability of admission interviews has been improved through the use of objective and structured approaches, there still remains the issue of identifying and measuring relevant attributes or noncognitive domains of interest. In this present study, we use generalizability theory to determine the estimated variance associated with participants, judges and stations from a semi-structured, Medical Judgment Vignettes interview used as part of an initiative to improve the reliability and content validity of the interview process used in the selection of students for medical school.</p> <p>Methods</p> <p>A three station, Medical Judgment Vignettes interview was conducted with 29 participants and scored independently by two judges on a well-defined 5-point rubric. Generalizability Theory provides a method for estimating the variability of a number of facets. In the present study each judge (<it>j</it>) rated each participant (<it>p</it>) on all three Medical Judgment Vignette stations (<it>s</it>). A two-facet crossed designed generalizability study was used to determine the optimal number of stations and judges to achieve a 0.80 reliability coefficient.</p> <p>Results</p> <p>The results of the generalizability analysis showed that a three station, two judge Medical Judgment Vignettes interview results in a G coefficient of 0.70. As shown by the adjusted <it>Eρ</it>²scores, since interviewer variability is negligible, increasing the number of judges from two to three does not improve the generalizability coefficient. Increasing the number of stations, however, does have a substantial influence on the overall dependability of this measurement. In a decision study analysis, increasing the number of stations to six with a single judge at each station results in a G coefficient of 0.81.</p> <p>Conclusion</p> <p>The Medical Judgment Vignettes interview provides a reliable approach to the assessment of candidates' noncognitive attributes for medical school. The high inter-rater reliability is attributed to the greater objectivity achieved through the used of the semi-structured interview format and clearly defined scoring rubric created for each of the judgment vignettes. Despite the relatively high generalizability coefficient obtained for only three stations, future research should further explore the reliability, and equally importantly, the validity of the vignettes with a large group of candidates applying for medical school.</p

Lack of trust in maternal support is associated with negative interpretations of ambiguous maternal behavior

Attachment theory assumes that children who lack trust in maternal availability for support are more inclined to interpret maternal behavior in congruence with their expectation that mother will remain unavailable for support. To provide the first test of this assumption, early adolescents (9-13 years old) were asked to assess whether ambiguous interactions with mother should be interpreted in a positive or a negative way. In our sample (n = 322), results showed that early adolescents' lack of trust in their mother's availability for support was related to more negative interpretations of maternal behavior. The associations remained significant after controlling for depressive mood. The importance of these findings for our understanding of attachment theory, attachment stability, and clinical practice are discussed

Do personality traits assessed on medical school admission predict exit performance?:A UK-wide longitudinal cohort studyA2

We thank the UKCAT Research Group for funding this independent evaluation and thank Rachel Greatrix and Sandra Nicholson of the UKCAT Consortium for their support throughout this project, and their feedback on the draft paper. We also thank Professor Amanda Lee and Ms Katie Wilde for their input into the application for funding, and ongoing support.Peer reviewedPublisher PD